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In this paper, we survey models proposed by Cleary, 
ThorndikeT Cole, Linn, Einhorn and Bass, Darlington, and Gross 
and Su for analyzing bias in the use of tests in a selection 
strategy. Several additional models are also introduced Oi;r 
purpose is to desc^be, compare, contrast and evaluate these 
models while, at the^same time, extracting such u^feful ideas as 
may be fourtd in these approaches* Several of these models (those 

N 

of Thorndike, Cole and Linn) are judged to contain operational 
contradictions because of their use of the wrqhg conditional 
probability within the context of the probabilistic structure. 
These models* deriving from a concept of grotip parity, are also 
shown to have highly objectionable practical implications. It / 
is suggested that the use of any of the^e^iiodels is cbntraindicated 
and that, indeed, the very concept of culture-fair selectiori» is 
unworkable. It is then suggested ' that t/ie necessary level' of 
compensatory treatment for disadvantaged^ persons can be guaranteed 



^This paper derives from material contained^ in a thesis sub- 
mitted by Nancy S. Petersen in partial'' fulfillm^tit of the requj.rer~ 
ments ^or the degree of Doctor of Philosophy 'atljthe University of 
Iowa, July, 1974. . ' 



only through the formal use of an appropriate model based on the 
Von Neumann-^Morgenstem theory of maximizing expected utility. 
Thre^ of the models studied (Cleary, Eilxhom and Bass, Gross and Su) 
^jare based on what we judge to be the correct conditional probability 
and are special cases of the Expected Utility Model, but each haa 
limited applicability. 
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^ Introduction 

Tests are befng used es/tensively by businesses and educational 
institutions for the screening of applicanrts fpr jobs or training 
programs. A major problem facing these institutions is how to 
eliminate cultural or racial unfairness arising from the ^se of 
tests in this process. There are many different definitions of 
what constitutes culture-fair selection, each implicitly, though 
unfortunately not explicitly, involving a particular set of value 
judgments with different implications for how selection should be 
accomplished. For each of these definitions a remedy has been pro* 
posed. Our purpose in this paper is to show that some of these 
approaches are inad^ijau^te to their task and that more complex analy- 
ses are required. 

Description of the Selection Process 

V The selection process can be chan?^cterized in the same manner 
for all selection models. First, there are individuals about whpm 
decisions are required. These decisions are to be based on infor- 
mation about the individuals. The information is proce^3ed by some 
strategy which leads to a final decision. The final decision ends 
the decision-making process by assigning the individual to either 
a select^ed or an outseleated group. The outcome 1^ the individual's 
performance after the assignment or, in other words, the consequences 

resulting from the decision. (Cronbach and Gleser, 1965, p. 18.) 

/r / ' 

mk strategy is a rule for making decisions. Each selection model 
reiSresents a strategy, the intent of which is to guarantee cultural 



fairness in the selection <]5rocess. Information is generally pro- 
vided by a test and we shall us^^si^ie* term test .to refer to ^all 
information-gat he ting procedures including Interviews and pl^ysi9^1 
m^surements. The overriding problem is the lack of agreeing 
as to the meaning of the term culture-fair selection. 

Each of the selection models or strategies we discuss ^can be 
characterized in the same manner. It is assumed that the*appli- 
cants to an educational institution, to a training program* or for 
employment can be separated into subpopulations because of an a priori 

belief that the assumed linear regressions within these 
subpopulations are different :^that is, the test (or predictor) may 
be more valid for some subpopulations thaa f6r others (different 
slopes), and/or for a fixed value of the predictor, the level of 

I 

criterion performances -'may differ (different intercepts), or that 
some differential selection criterion is appropriate for various 
subpopulations. Alternatively, these subpopulations may be differ- 
entiable primarily because of public concern with what is going on 
in them and a public need , to verify that 

all subpopulations are being handled "fairly'*. Specifically, it is 
assumed that initially a ctiterion score (Y), 'as well as a predictor 
or test score (X), is available for members of each subpopulation, 
implying that in the past all applicants have been admitted or em- 
ployed .regardless of their score on the test. A minimum level of 
satisfactory criterion performance (y*) is de^termined. The number 
of applicants that can be selected is ^eclfied.'' If there ±0 no 



constraint on*irhe number oT applicants that can l^e accepted, then 
the selection situation is' referred to as quota-free selection ; if 

r • 



only a fixed proportion of the applicants can be accepted, then the 

selection situ^tJ-on is. referred to as restricted selection ^ A cMt 

score' (x*) on the pi:^dictOT neecfs then Co be calculated for each 

subpopulation so that the definition of culture-fair selection 

specified by the -particular model is satisfied. In the case of 

multiple predictors or tests (X- , X^, X ); the cut score will 

•» i z m- 

be determined on the variable formed by the usual .least squares 
linear combination of the predictor variables, the future, appli- 
cants with a test score above the predictor cut score for their sub- 
population will be selected and applicants with a test score 
below the predictor cut score for" their subpopulation will be rejected, 

This selection strategy presupposes that an^cceptable criterion 
varia-ble is available. The inappropriateiiess of the criterion^vari- 
able will not be treated in this paper, although this may be the 
most important problem. Thus, the following discussion of selection 
fairness will be based on the premise that the available criterion 
score is a relevant, reliable and unbiased measure of performance 
for applicants in^ each subpopulation. All previous contributions 
\xi this ^rea have implicitly made these assumptions. Caveat emptor ! 

The Regression Model 

The Regression Model for "test bias" has been well stated by 
Cleary (1968): 

A test is biased for members of a subgroup of the popula- 
tion if, in the prediction of a criterion for which the * 
test was designed, consistent non-zero errors of predic- 
tion are made for members of the subgroup. In other words, 
the test is biased .if the criteriorr score predicted from 
the common regression line is consistently^ too high or too low 
for members of the subgroup. With this definition of bias, 
, there ma^ be a connotation of "unfair," particularly if the 
use of the test produces a prediction that is too low. [p. 115.] 
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Here, standard regression theory with the minimisation of mean- 
squared error is used to. provide predicted criterion scores* 
At the tninimum level of -satisfactory criterion performance (y*) , 

»* it* ^* ^ 

, y. = + B^x^ = ... = ag +-6gXg , ' (1) 

whfere a^, g^, and represent the intercept, slope and predictor cut 
' score for subpopulation 7r^(i = 1, g) , respectively. If the 

regression lines are identical in each subpopulation, then the.use.,of 
the common regression equation to select applicants with the highest 
^ predicted criterion scores is considered fair. ' ^ 

Using the Regression Model,. 'and assuming thdt the parameters 

' ^ ' * 

^^1' ^1^' ^^2* ^2^ ^^g' ^g^ known precisely, a decision 
maker can be assured that the average pi;edicted criterion score, 
given the" available predictor variables, will be a maximum for the 
applicants selected and, incidentally, a minimum for the applicants 
rejected. Using the Regression Model, the applicants can be assured 



that the selection ^procedure is '*fair'*^to individual members of each 
subpopulation in that criterion performance is not systematically 
under or overpredicted for members of any subpopulation. Or to put 
it another way, the Regression Model says that if two' applicants are 
being considered for one -post,, then that applicant having the highest 
predicted performance would be selected with^predlctlon being made 
on the .basis of subpopulation regression. 
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Jhls definition of culture-fair sele(ttlon assumes fairness 
is achieved if the applicants with the highest predicted criterion 
scores, using separate regression equations within ,«(xbpopulations, 
are selected. From this point of view, selection Is fair' if and 



only if it is based on the best prediction available. . Thus,' opti- 
mal prediction and fairness (lack of bias) are taken to be strictly ^ 
equivalent and the procedure adopted is that which maximizes 
expected performance for each individual and hence overall. 

To illustrate, suppose the applicants to an institution 

* < - 

can be divided into two subpopulations referred to as subpopu- 

lation TT and ^ubpopulatlon ^ Now, refer to Figure 1. In 

* * , Figure 1(a), the regression lines for the two subpopulations have 

the same slope but different intercepts. In Figure 1(b), the 

* regression linps for the twp subpopulations have different slopes 

• ' ' ' ' ^ , \* 

and different intercepts with the point of ^intersection ourside th^ - 

, . - rAnge of possible test scores. In each of these' situations, suppos^A 

■ ' ' ' ' ' ' //^ . ' . ^ ' - 

the common tegressioti line (t[ ) for the total' applicant "papulation 

.... ^ '-^^ . • . 

were used for ' predicting criterion scores for alj applicants rather 

' separate within-subpopulatioA regression lines, then for , any 

given test score,, criterion scores for .subpop'ulation tt^ would be 
. \con8lstently underpredicted ; an4, flieref ore, this subpopulation would 
;be discriminated against by t;he test.- ' In .Figure 1(c), the ' ' ' 
. . ^ ' regression lines for , the two subpopulations again have different 

^ ';slbpes and 'different intercepts, but the point of intersection "is / 
' . inside the rantge of possible 'test scores. ■ If the common regression / 

line were used for predicting criterion scores, thep some individual^ 
from both subpopulations would be discriminated against. At point 

on the test, the criterion score for a member <^f subpopulation ttJ 
would be undei-predicted and^ at point on the test, the criterio^ 
score for a member of subpopulation rr^ would be uhderpredicted. ij 
i ' the special case in which the regression lines fir the two subpopu-| 

lations coincide, an applicant's predicted criterion score is the 

ERIC . 3. 
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Figure 1 

Iliustration of Culture-fair SelA:tion 

as Defined by the Regression Model 
Criterion (Y) 




Figure 1(a), Supopulations with parallel regression 
lines but different intercepts* 



Criterion (Y) 




Test (X) 



Figure 1(b). Subpopulations with different regression 

lines. Point of intersection outside range 
of possible test scores. 



Figure 1 (cont'd.) 
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Figure 1(c). Subpopulations with different regres- 
sion lines. Point of intersection ^ 
inside range of possible test scores. 
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e> regardless of group member shl'p , for ariy given test score. 
^The Regression Model is the most widely used model' of selection 
fairness within the predictive context. It Is a sttaightf orward 

i 

application of minimum-mean-squared error theory. * Xt ha^t been 
used in a number of empirl<^al' studie^ (e.g.. Clear y, 1968; Bowers, 
1970; Temp, 1971) and it has* beenr-rosic in the conceptualizations 
and discussions of selection fairness that may be found in Anastasi 
(1968)^ Guion (1966), Bartlett and O'Leary (1969), Einhorn and Bass 
(1971), Linn and Werts (1971), Linn (1973), and Schmidt and Hunter 
(1974). • "* . ' 



\ 



The Constant Ratio Model 
1 



. Thorndike (1971) suggests that in^a^^tudy of culture-fair 
selection we 'should consider the implications for the proportions 
of a^pplicants admitted from each subpopulation as well as the 
implications of the within-subpopulation regression lines as was 
suggested by the Regression Model. He demonstrated that if a 
test has equal regression lines for each subpopulation but the 
discrepancy between subpopulatlons on the test differs from the 
discrepancy between subpopulations on the criterion, then' the use of 
the selection, strategy* ^.niplied by the Regression Model, ^ 

^ ' 

which is "fair" to individual members of the group scorip^g ; 
lower on the test, is "unfair" to the lower [scoring] group 
as a whole in the sense that the proportion qu^ified on 
the test will be smaller, relative to the higher-scoring 
group, than the proportion that Ijill reach any specified 
level of criterion performance. [p. 63.] 

Thorndike proposes that in a fair selection procedure 

the qualifying scores on a test should be set at levels that 

will qualify applicants in the two groups in proportion to 

the fraction of the two groups reaching a specif led .level 
Of criterion performaape.^ [p. 63.] 
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This definition assumes that the selection procedure i^ fair if ^ 
applicants are selected' so that the ratio of the proportion selected 
to tfie proportion successful i>w the^same in all subpopulati<ms;' hence 
the reference to it. as th^ feonslsant Rat^ Model. Therefore, given a 
minimum level of satisfactory criterion performance "(y ), a selection 
procedure is considered fair when 

Prob (X >i X* I ir ) p/ob (X >. x* | ir ) ■ 

R_ — — Q 2— 

— I . • • • ^ » 

?rob(Y >jf . P^ob(Y>~-y lir^) 



f 



A^ere R 'is a fixed constant for all fsubpopulations tt , and x repre- 

sents th^ predictor cut score for su^population 7r_^(i = 1, g) . 

V 

It should be noted that, Thorndike did not give a formal statement //)f ; 

' 'i ^ ^ ' " 

a model, only a general prescription. The explication of the modeT, 

. ' \ . ^ ' 

as given above, is due to Cole (1973). 

• t * ' 

To illustrate, refer to FigureA^2- (Adapted from Thorndike, 

1971, p. 66.) Assume the applicants to the institution were divided^ 
into .two subpopulations, and ir^. Figure^2(a) depicts the 
situation which Thorndike refers to as being "fair" to •indi\j'idual 
members of the minority population ir^-but "unfair" to the minpripy . 



^"""to 'simplify the diagrams in Figure 2 and Figure 4, it is 
assumed that (1) th« variables X and Y have a bi^arlat-^ normal distr'i- ' 
bution in each subpopulation, (2) the correlation between X and Y • 
(r ) is positive, and (3) the standard deviation of the test ^s^) , 
. the "standard deviation of the cEitei^ion^(Sy) , and are constant - 
for each subpopulati6n. furthermore, the predictor cut score (X2) . 
for the majority' population is. choseti to be on the regression line 

"ic "k 

(i.e., fdr subpopulation ir^^ given X = x^, then^ Y = y ). The predictor 
cut score (x*)' for the minority population is then adjusted 
accordingly^ , ' ^ . S 

- 12 



Figure 2 

lHustratiori of Culture-fair S^election 
as Defined by the Constant Ratio ^Model' 



i 



Criterion (Y) 




Test (X) 



Figure 2 (a;. Subpopulations with common regression 
line. Mean difference on test is not 
^ equal to mean difference on criterion. 
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Figure 2 (contM.) 



Criterion (Y) 




Test (X) 



Figure 2(b). Subpopulations with parallel regression 
lines. Mean difference on test eqti^ls 
mean differ^ence on criterion* 



Criterion (Y) 




Test (X) 



figure 2(c). Subpopulations with parallel regression 
lines. Identical criterion score 
distributions- 
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population as a whole. The regression is identical in each subpopu- 
/ 

lation;^ thus the test would be consi<3ered fair ac^cording \to the 
Regression Model if all individuals/ regardless^ of group mepbership, 

who have test scores greater than or equal to x* are selected* Note 

2 

that the mean of X in gubpbpulation ir 'is less than in subpopulation 

1 f 

* \ 

that this difference is greater than the corresponding ' , 

difference on the criterion measure. If only those applicants with 

predicted criterion scores equal to of greater than y* were selected, 

then approximately 50% of subpopulatiofi i;^ would be accepted and 

approximately 50% would be successful, but essentially no member^ of 

subpopulation t;^ would be, accepted, yet appro^^iSioately 10% of the - 

members of subpopulation would havQ^been^^ucte^ Thus, if 

X2 is used as the predictor cut scor^ for each subpopulation, the 



^ test discriminates against subpopulation tt^ according to the Constant 
Ratio Model, In this situation, to make the selection piocedure fair 

I 

according to the Constant Ratio Model, the members of subpopulation 

it 

with test scores greater than, or ^qual to X2 would b© accepted, 
and members of subpopulation ir^ with test scores greater jthan or 

^1 



equal to x^ would be a^ccepted. It is important to note, ^however, 
that if the difference in the group means on the predicoor ^-is less 



than that on the criterion measure, then application of / the Constant 




Ratio Model will give the lower cut score to subpopulation tt^ rather 

n to TT^. 

W In figure 2(b), the regressipn lines are parallel and-j^he^ 
difference between means on the tjest is the same as the difference 
between means on the criterion. The ratio of the proportion quali- 
fied on the test to the proportion successful is the same for each 

subpopulation* This strategy is fair according to the Constant 

i 
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Ratio Model, but not according to the Regression Model. Vft^fen 
ther« are differences in the distributions of the criterioh 
scores, the strategy is fair according to both the Const^At Ratio 
Model and the Regression Model, only if the validity is perfect and 

the regression lines are the same for each subpopulatioj. 

- -^K. I 

Iii Figure 2(c), the regression lines are parallel >^nd the 
distribution of criterion scores is the same for both .^ubpopulkt ions. 

If y* represents the minimum level of satisfactory criterion per- 

' ' "- 

formance, then the same selection strategy would be considered fair 

' / 

by both the Regression Model and the ^Constant Ratio Model. The 
.institution woul^^accep)^members of subpopulatipn who had test 

scores ^^reater than or equal to x- and it would accept members 

/ 

of subpopulatiori ir^ who had test scores greater th^'n or equal to 



In many applications, the mean^ criterion scofe of the min<5rity. 
population it will be less than in the majority pbpulatioa^.TT^- -and 
that difference will be less than the difference jf the ^edictor 
means, in which case an acceptance procedure based on the Constant 
Ratio Model will almost always atcept applicants from the minority 
population it who do less well oii the criterion, on the average, 
than applicants from the majority population tr^.. This feature 
explains the attractiveness of thi3 model. ♦ 

The Constant Ratio model and the Conditional Probability and 
Equal Probability models, to follow, have been described (Sawyer^j 
Cole and Cole, 1975) as Group Parity models in that they focus on 
fairness t6 groups rather than to individuals. By contrast other 
models studied focus on the individual. 
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The Conditional Probability Model 

Cole (1973) proposed a fully explicated criterion for^ culture- 
fair selection based on the conditional probability of being selected 
given satisfactory criterion performance; l^ence the reference to 
it. as the Conditional Probability Model. Cole argues that all appli- 
cants who, if* selected, are capable of b^ing successful should be 
guaranteed an equal, or fair, opporliunity to be selected, regardless 
of their group membership. ♦ 

The basic principle of the conditional probability selection 
model is that for both minority and qiajorj^ty groups wfiose^ 
mem.bers can achieve a satisfactory criterion score [Y > y ] 
there should be the same probability* of selection regardless 
of group membership. [p. 240.] 

Therefore, given a minimum level of satisfactory criterion perfor- 

mance (y ) , a selection J)rocedure lis considered fair when 
/ 

K » Prob(X > x*|y y*, tt-) = = Prob(X >^ x*|Y >^ y*, tt ), 

, ^ 8 8. 

where K is a fixed coni^tant for all subpopulationp tt^ and x^ repre^,^,^ 
sentjs tfie predictor cut score for subpoptilation ^j^(i " 1> 

Figure 3 is an illustration of a hypothetical bivariate 
distribution of test and criterion scores* Indi^/^duals falling in 
region II hav'e test scores less than the predictor*-^ cut score (they 
would be rejected); yet, if selected, they would have satisfactory 
criterion performance. Such individuals are referred to as false 
negatives > False positives are those individuals with test scores 

greater than the predictor cut score (they would be accepted) but 

1 

with unsatisfactory criterion performance • 5uch individuals fall in 
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Figure 3 

A HypoSietical Bivariate Distribution 
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region IV. The assignmertt of an individual to either region II or 
IV is an incorrect decision (error). Correct decisions are made 
^ for those individuals assigned to regions I and III. (Linn, 1973, 
pp. 152-153.) 

The emphasis in the Conditioual Probability Model is on the 
number of applicants in region *I in relation to the number of 
applicants in regions I and II combined, whereas the emphasis in 
tjie Constant Ratio Model is on the number of applicants^ in regiozxa^^^ 
I and IV combined in relation to the number of applicants in regions 

f t ' /" 

I and II combined. 

Figtir^ 4 contrasts the Regression ModeL^ the Constant Rati^ 
Model and the Conditional Probability Model for the situation* in* 

whtch the regression is identical for each isi&population, but the 

s 

mean test score and the mean criterion performance is less for 
Members of subpopulatlon tt^ than for members of subpopiUation 
^ 1^2* (See comment in r'feferenqe to Figure 2.). In Figure 4(a), 
all applicants, ifegardless of group membership, who havje test 
scores greater than x , are accepted. Using this selection strategy, 
the selection procedure would be considered fair according to 





Linn (1973, p. 153) stated that a tes£ was fair according to 
Thorndike^s definition of selection fairness (the Constant Ratio 
Model) if the number oj individuals in region II equals the number 
of individuals in region IV. (See Figure 3.) Strictly speaking, 
the Constant Rant© Model does not require equality of region II and 
IV, However, the .^odel will be satisfied and equality of regions 
II and IV will occur if and only if the selection-success ratio R 
[Equation (2)] equals 1 implying Prob(X 2. x^Itt^) = Prob(Y >^ y (tt^) 
for each subpopulation tt^^^I For purposes of heuristic comparison 
among models, we shall assume that this assumption holds. 
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' Figure 4 

A'Constrast of "the Regression, the Constant Ratio, and 
the Conditionai'Frobability Models 



Criterion (Y) 




Tfest (X) 



Fi^gure 4(a). Subpopulations with commort regression . 

line. Selection strategy, fair according 
to Regression Model. 
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Figure 4 



Criterion (Y) 




Test CX) 



Figure 4(b). ^ Subpopulation^ with common regred^ion line. 
Selection strategy fair according to 
Constant Ratio Model. ' - • 



Criterion (Y) 




Tes^ (X) 



Figure 4(c) . 



Subpopulations with common regression line* 
Selection strategy fair according to 
Conditional Probability Model. 
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the Regression Model, .la Figure 4(b), applicants from subpopu- 

^latlon TTj^ are accepted if they have test scores greater than x^, and 

^. api^licants from subpopulation ^2 accepted if they have test 

scores greater than x*. The ratio (I + IV)/(I + It) is constant 

for each subpopulation. (Refer to Figure 30 Thus, using this 

selection strategy, the test is considered fair according to th*e * 

Constant Ratio Model. In Figure 4(c), the predictor cut score for 

subpopulation is fuid for subpopulation 712^ the predictor cut 

score is x*. Here the 'ratio I/(I.+ II) is constant for each subpopu- 

latlon (reS^er to Figure. 3). Using this selection strategy, the 

test is considered fair according to the Conditional Probability Model 

} Note that as with the Constant Ratio Model, a selection strategy 

basecj on the Condit^ional Probability Model will almost always accept 
applicants from subpopulation tt who do less well on the criterion, 

on the average, th^n^> applicants from subpopulation. tt . Also note 

y ' ' ^ 

that ii an applicant from subpopulation tt^ i$ predicted to do just 

as well on the criterion as an applicant from subpopuXation then 

a selection 'strategy which is fair according to the Regression Model 

will consider the two applicants ♦equally -desirable candidates for 

admission. However, »a selection str^ategy which is fair according 
to the Constant Ratio Model will^ive preference to the applicant 

from subpopulation tt^ and a*selectdon strategy which is fair accord- 
ing to thfe Conditional Probability Model will given even greater ^ 
preference to that applicant. Note also that this preference for 

subpopulation tt, will hold even if the "minority" population happens 
^ • 1 ' J 

to be majority subpopulation tt^, as will be dis(iussed later. The 
remarks in this paragraph can be sirbstantiated by noting that the 
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Cole criterion can be written as 



Prob(X 1 X*/ Y >. y*) = 



Prob(X ■> X ) 



Prob(Y > y*) 



Prob(Y >. y*(X 



-1 ■ 



The first factor on the right is the Thornd ike, ratio and the secdnd 



e^c 



factor will be smaller for subpopulation ir^ than for under^the 
spercif ic conditions assumed earlier. Thus, for the Cole defini- * 
tion of selection fairness to be satisfied the cut score x for 
subpopulation ir^ must be lower than that value required to satisfy 
the Thorndike definition. /' ^ 

The Equal Prpbability Mod^l 

In the "usual selection situation^ the "given" infortnation for 
each applicant is not his future state of being (success or failure)- 
in relation to the criterl^ variable, *but rath6r, his present ob- ^ 
served standing on the predictor variable. Thus, from one point 
of view, it would seem reasonable^ to propose a definition of culture- 
fair sel^ctifon based on the conditional probability of success given"* 
selection. One might argue that ali applicants who are selected should 
be guaranteed an .equal, or fair*, chance of being successful, regard- 
less of group membership. Such a model for selection was described 

by Linn (1973, p. 153) and shall^ow be referred to as the Equal 

i. 

. t 

Probability Model. 



+ 

Linn (1973, p. 153) described the Equal Probability Model but 
referenced it as the traditional psychometric approach suggested by 
Einhorn and Bass (1971). The definition of cultote-fair selection 
suggested by Einhorn and Bass, to be called the Eqiial Risk Model, 
will be discussed later in. the paper. At this point, it is enough 
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According to the Equal Probability Model, within the selected 
group from each subpopulatlon tt^, the proportion of successful 
performers should be the same. Therefore, given a minimum level 
of satisfactory criterion performance (y ), a selection procedure 
Is considered fair when 

Q - Prob(Y >y |X >. x^, tt^) ... » Prob(Y > y ^jX > x^, tt^) , - 

. (4) 
* 

where Q Is a fixed constant for all ^subpopulatlons tt and x, repr'e- 
sents the predictor cut score* for subpopulatlon 7r_j^(l 1, g). 

In reference to l^lgure 3, the emphasis In the Equal Probabil- 
ity Model Is on the number of applicants In region I In relation to 
the number of applicants In regions I and IV combined. In reference 
to Figure 4, the selection strategy depicted In Figure 4 (a) is fair, 
according to the Equal Probability Model, when members of subpopula- 
tlon 7T and members of subpopulatlon tt^ who are predicted t;o do 
equally well on the criterion' are considered equally desirable 
candidates' for admission. Clearly, a selection strategy dictated 
by the Equal Probability Model will not typically coincide. with one 
derived. from either of the three preceding models. ' Thus, the prac- 
titioner is faced with the task of choosing from among four "attrac* 
tive" models. How should this choice be made? 



to. note that in the Equal Probability Model the conditioning is on 
X >_ while in the Equal Risk kodel the conditioning is on X » x^. 
It should be emphasized that. the Equal Probability Model was not 
4)roposed by Linn, but was only discussed b^ him as an academic exer- 
cise. By contrast both the Constant Ratio and jCondltional Probabil- 
ity ihodels have been recommended for current adoptiori by Cole. 
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The Converse * Constant , Ratio Model 
The last three models described (the Constant Ratio Model, 

* ' • • » 

the Condttlo&l Probability Model and the Equal Probability Model)* 
presented definitions of culture-fair selection stated in terms of 
sudcess and selection probabilities* Conceptually^ it seems just 
as reasonable to explicate the fundamental concept cff each approach 

• ■ \ / 

by exhibiting concern for the rejected^ and/or tinsucceasful appli- 
cant. Thus, the following three models-j^for cuiture-fair selectidit 
will be restatements of the^ previous three toode Is in terms of 
failure and rejection. ' ^ " 

Recall that the Constant Ratio Ifodel compares selection rate witfr 
success rate in each siibpopulation* The emphapls is on the proportion 
of at)plicant:s who are selected in relation t^ the proportion of apf)ll- 
cants who are successful* However, one could conceivably conaidei? it 
just as important or necessary to consider the implication^ for the 
proportioa^f applicants rejected in each subpopulation* One could 
propose that the cut scores on a test should be set at levels that will 
reject applicants in each siibpopulation in proportion to the fraction 
oi" ^ch suBpopulation failing to reach a specified minimum level of 
criterion performance. -Such a selection strategy will be referred to 
as the Converse Cpnstant Ratio MWel* If we are particularly concerned 
with a subpopulation tt^, then we would want to be sure that the 
ratio of the proportion rejected to the proportion failing be no 
tfliore than In any other subpopulation, whereas with the original 
model we would want fhe ratio of ^ the proportion selected to the 

. / , ■ . • 

proportion successful to be not less than in any other subpopulation* 
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The definition assumes that a aelection procedure is fair if 
applicants are rejected so that the proportion rejected to the 
proportion unsuccessful is, the same in all subpopulations. Ther*efore> 
given a minimum level of satisfactory criterion performance (y ), a 
Selection procedure is considered fair when 



Prob(X'< xJirJ 

R « ' ^ 



Prob(Y <'y*|7rp , i 



Prob(X < X lir ) . 
g g 

Prob(Y < y*|7r h 



(5) 



wher^ R> is a fixed constant for all subpbpulations u^. and repre- 
sents the predictor cut Vcore for subpopulatllon 7r^(i ^1, g), 

■■■ -L . 



The above relationship can be rewritten as 



1 - Prob(X >. x^Itt^) 
1 - Prob(Y >_ y*|Tr.) • 

[Prob(Y > y*|Tr^):]~-'- - Prob(X > x* | f^) [Prob (Y > y*|Tr^)]~-'- 
[Prob(Y > y*}Tr^)] 

[Prob(Y > y*|Tr^)]"^ -'r 
[Prob(Y > y*|u^)]"'^ - 1 



-1 



- 1 




where 



R » Prob(X > x*f7r.) [Prob(Y > y^lir.)]""^ is the value to be 



equated among subpopuiations for ' selection fairness as specified bj^ the 



Constant Ratio Model. NoW, suppose we have specified a minimunj level 
of satisfactory criterion performance (y ) and a selection-success 

■ v -t',,^ 

ratio (R), then a predictqr^ut score can be determined for ^ach 
^^jApopulation 7r^(i « 1, g). GJLven the values y , R, and x^^ 

^''^^ll 



the rejection-failure ratio 



be constant for each subpopulation 
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if the following conditioa-ls satisfied: 



[Prob(Y > y*{Tr^)r^- - R [ProbtY > 7*1^)]"^ - R. 
[Prober > y*JirJr-^ - l ' [Prob(Y > 7*\itjf^ - 1 



for 1, 5 • 1, . • • » 8- 



The above condition will be satisfied if and only if either (1) R = 1 
implying Prob(X >, x*|Tr^) « Prob(Y >^ y^jir^) for i « 1, . . g, or " 
(2) Prober > y*^\it^) » PrpbCl > y*|Tr^) for i, j - 1, . . . , g, but not "'^ 
generally^ If either case (1) or case (2) obtainfs, Ithe same set of 

it 

predictor cut scares (1 « 1, ... , g) is considered fair according . 
to both the Constant Ratio Hbdel an4 its converse^ but» otherwise, 
the strategies will differ. In Thomdike's illustratfBfl (1971, p. 66), 
he set R. » 1, though he did not indicate that this was ^eqtiired by ^ 
his model. Only by reference' to various real applidltions might ^ 
we be convinced that ^ »'l will be a commonly accep^table value. 
However, with restricted selection, It^ is not generally Cposisible |;g 
simultajaeously satisfy this condition and the selection constraint." 

Consider carefxilly the force of the following argument. If fairness 
to subpopulation tr." demands that the selection-success ratio (R) 
be the sShe for any other subpopulation ir., then, with identical 



logle> falrnees to subpopulation ir. demands that the rejection- 
failure ratio (R) be the same for any other subpopulation ir^, and 
the two specifications cannot both be satisfied* This strikes us as 

. ■ . . 4 

being a logical contradiction; however, each reader must make a> 
persoiial judgment on this- point. 



I 



■A 



eric; 



.25 



The .Converse Conditional Probability Model .• . 
The' Conditional Probability Model Is based on tbe conditional 
probability of being selected given satisfactory criterion perfor- 
tnance. ^ The emphasis ^is on the proportion of potentially subcessful 
applicants vho are selected. However, one could argue instead that 

potential failures should be rejected in 
no greater .percentage in adj^subpopulation.* J|e shall ^ 
label this selection strategy the Converse Conditional Probability 
Model. . - 

' The Converse Conditional Probability Model is based on the 
conditional probability of being rejected given unsatisfactory 
criterion perfomtance. Therefore^ |given a minlnmm level of satls- 
fdifetory criterion performance (y*) , a* selection procedure is 
considered fair when \ . / 

K = Prob(X < xJy < y , tTj^) * ... " Prob(X < x^l^ < 7 . ^g). 

>- . .. (6) 

wh^re K is a fixed constant for all subpopulations tt.* and x- repre-^ 
6e^ts the predlCt-ox: cut score for subpopulation tt^jJ^t-- 1, ...» g). 
•The ^bove relationship .can be rewritten as 



K «• Prob(X < x^, Y < y*|Trp[Prob(Y -f y*|Tr^)]~-'- \ ■ j 




- {[Prob(X>x*, Y>y*|Tr^) + Prob(X<x*, Y<y'^|Tr^)]-Prob(X>x^; Y>y"|ir^).} 
[1 - Prob(Y > y*|n^)r^ 



2S 

T 
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- {[Prob(^x*, X^*K^) + Prob(X<x*, Y<y*|irj.)]{Prob(Y>y*|ir^)]"^ 

- Prob(^x*, ?^*|ir^)[Prob(Y>7*|ir^)]"^}{[Prob(Y>y*jir^)]"^-l}"^ 

- {[Prob(^x*, Y>7*|irp'+ Prob(X<xJ, Y<y*|ir^)] [PrQb(Y>y*|ir^)]"^- K) 



{[Prober >y*|ir^) 3'^ - X)"^ 



where K « Prob(X > x^, Y >^ y jir^)[Prob(Y > y (ir^)]"^ is the value to 
be equated among subpopulations for selection fairnejas as specified by 
the Conditional Probability Model". NdWT-s^uppose the decision maker 
has specified a minimxmi level of satisfactory crit^^n performance 
(y ) and a constant conditional probability of selection given success 
(K), then a predictor cut score x* can b6 determined for *ach subpopu- 
latlon tr^»(i » T, ^^..^ g)* Given the values y , K, and x^^, the 
conditional probability of rejection given failure tK) will be 
constant for each subpopulation tt^ if the following condition is 
satisfied: ^ 

{IProb(X > X*, Y > y*lTr^) + .Prob(X < x*, Y < y*|Tr^)] 

[ProbCY > 7*\jf^)r^ - K} {[Prob(Y > y*|ir^)]"^ - p'f- 

" {CProb(X i x*, Y > y*|ff^> + Prob(X < xj, Y < y*(ffj)] 

[Prob(Y > y*kj)]'"^ - K}{[Prob(Y > y*\t^)]~^ - 1}"^ 

for 1» j - 1 g. 

) 

29 



- • . " 27 

This condition will be satisfied if and only if Pyob(Y^ lTr^)-Prolj(Y>y jir^ 
'and Prob(X < x*, Y < y'\i<^) " Prob(X < x*, Y < yVj) for i* 
J « i, g» a most unlikely itgte of affairs. In that case, 

the teme selection strategy, ' the same set^ predictor cut 

Scores x^,(i « 1, ...» g), is considered fair according- to both 
the Conditional 'Probability Model and the Converse Conditional 
Probability Model, but, otherwise, the selection strategies will 
differ. It does not seem at all apparent to us whether for the 
particular subpopulatioil that is of public concern it is more 
ImpQrJtant ^o keep the conditional probability €i rejection (given 
potential ^failure) low or the conditional probability of acceptance 
(given potential success) high. ' Unfortunately, the two criteria are 
based on different conditioning events and hence are contradictory. 



The Coaveyse Equal Probability Mpdel 
The Equal Probability Model is based oil the conditional proba- 
bility of success given selection. The emphasis is on the proportion 
of the selected applicants who are success-ful. However, one could 
propose that all applicants who at^ rejected should have the same 
probability of being a failure, regardless of group membership^ Such 
a selection strategy will be labeled as the Converse Equal Probability 
Model." . ' - 

The Converse Equal Probability Model is based on the conditional 
probabi^tg^of faijure giyen rejection. Therefore, given a'mlnimum 
level of sa^tisfactory criterion performance (y ), a selection - 
procedure, is considered fair when 



Q * Prob(Y < y |X < x^, ir^^) « " Prob<Y < y |x < x^, t^) \ 

30 ' 
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where Q.iB a fixed cons tan t for all subpopuiatlons ir^ and repre- 

>» 

^sents the predictpr cut score for subpopulation Tr^<i * 1% g)« 
The above relationship can be rewritten as 

Q « Prob(X < X*, Y < y*l7r^) [Erob(X < xjlir^)]"''- 

« {{ProbC^x*, Y>7*|uj^)+Prob(X<x*, Y<y*|7r^)3-Prob(X>x*, Y>y*(7r^)} 
[1 - Prob(X > x*|7r^)]"-'- 

• {[Prob(&x*,. Y>y*lTr^) + Prob(X<xJ, Y<y*(7r'^)l [Wb (X>x*(7rj^) j""^ 

-Prob(X>x*, Y>y*jTr^)[Prob(^x*|7rp]""'"}{[Prob(X>^^ l}"""" 

« {[Prob(X>xJ, X>3^*|Trp + Prob(X<x*, Y<y*|Tr^)3 [Prob(X>x*l7r^) j"''"- Q} 

{IProb(X > xj^lir^)]"^^ - ir^ , ^ ' . — 

where Q « Prob(X > x*,. Y y*|Trj) [?rob(X 2i 3C*|7r^ ia the value to 
be equated among subpopulations for select^Loi/^almess as specified by 
the Equal Probability Model. Again, suppose vlfe -have specified a 
minimum level of satisfactory criterion performance (y*) and a constant 
conditional probability of success given selection (Q), then a 
predictor cut score x* can be determined for eaqh subpopulation 

(i » 1, g).^ Given the values y*, Q, and x*, the conditional 

probability of failure given ^rejection (Q) will be coiastant for each 
subpopulation ir^ if the following condition is satisfied: 
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*. ^ 



{[Prob(X >. x^, Y > y (77^) + Prob(X < x^, Y < y |t7^)] 
tProb(X > x*iTr^)]"^ - Q}{[Prob(X > x*|ir^)]'^ - 1}"^ 
. - {[Prob(X > X*, Y > y*|uj) + Prob(X < x*. Y < 7*^^)] 

tProb(X > x*|tTj)]"-'- - Q}{[Prob(X > ^*\^pf^ l)"""", 

for 1, j = 1, g. — 

This condition will be satisfied if and only if v 
Prob(XL»ilV!- Prob(^x*|Tr^) and Prob(X < x*, Y < y*|Tr^)'« Prob(X < Y < y*kj) for i, 

J « 1, g, but not getierallj^* In that ca6e, the same selection 

strategy 9 the same set of predictor cut s^res x* (i « 1, • ••• 8)> 
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f Is considered fair according to bath the Equal Probability Model 

and the Converse Eqtial Probability^ llodel, but, otherwise, the selection 
Strategies will differ. . - 

Figure 5 compares the Constant RaCdLo. Model, the Conditional 
Probability Model, the Equal Probability Model and the three . 
**converse'^ models. Representing proportions $s areas, we see the 
Six ratios for the six models. Unfortunately, there are indeed 
six^ models, each seemingly attractive and each paired with an In- 
, distinguishable converse* Which is the appropriate model? Or, 
indeed^ is any, one of them acceptable in any common situation? 

The Equal Risk Model 

Einhorn and Bass (1971) proposed a model for culture-fair 
selection which takes itito account, for each subpopulation, the 
probability of success associated with an applicant's test score 
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, Figure 5 

A Comparison of Six Models for Culture-fair Selection 



Criterion (Y) 



/\ II 


I } 


/ ni 




s , 








X 





Test (X) 



The cut score on the test (x ) is determined so that the 
ratio (as specified by a particular model) is the same 
for all subpopulations. 



Model 
Constant Batio 
Conditional Probability 
Equal Probability 
Converse Constant Ratio 
Converse Conditional Probability 
Converse Equal Probability 



Ratio 
(I + IV) /(I + II) 
1/(1 + II) 
I/<I + IV) . _ . 
(Ill + II)/(III + IV) 
ni/(III + IV) 
TLlliXi-l + II) 



31 



rather than just the applicant's predicted criterion scor'e as 

suggested by the Regression Model. Their model Is based on a ^ ■ ' 

definition of selection bias given by Gulon (1966). Gulon stated that 

'unfair [test] discrimination exists when persons with equal' 
, probabilities of success on the job have unequal probabilities ' 
of being hired for the job. tp. 26*] 

Th^ objective of this model is not simply tp accept those persons 
who are predicted, in the sense of best point estimate, to be above 
a specified minimum point on the criterion, but rather to accept those 
persons for whom this prediction can be made with a specified degree 
of confidenq^^. The problem th^n becomes one of finding a cut score 
on the predictor variable so that the criterion score for persons 
with test scores greater than the cut score will be above the* 
minimum acceptable criterion scores with probability at. least equal 
to some specified value* Furthermore, this model specifies that this 
probability (oj, conversely, risk) must be the same in all sulj2fiP«^ 
lations; hence the reference, to it as 'the Equal Risk 'Model. .* 
Therefore, sjrmbolically, at the minimum level of satisfactory criterion 
performance (y ), the Equal Risk Model requires that the predictor 
cut scores Xr^(i « 1, g) be determined so that 

r 

Z » Prob(Y >. y |X « X,, ttJ «' ... « Prob*(Y >. y |X « x , tt ) , 

where Z is a fixed constant probability of success *for all subt)opu- 

« 

lations 7r^» ♦ . " . 

To Illustrate, again suppose the applicants to an institution • 
can be subdivided ii^to two subpopulatlons, ^7r£ and ir^* ^ Refer' to 
'Figure 6. (Adapted from Elnhorn and B^ss, 1971, pp. 265, 267.) 
Figure 6(a) shows the relationship between a predictor (test)' 



Figure 6 

Illustration of Cpiture-fair Selection 
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us Defined by thei Equal Rl^k Model 



V ' 



Criterion ^ (Y) 




Test (X) 



Figure 6(a)* Conditional distribution of criterion 
on test showing risk level. 



Figure 6 (cont'd.) 



Criterion (Y) 




Test (X) 



Figure 6(b>« Subpopulations with commcm regression 
line but different standard errors of 
estimate* 

Criterion (Y) 



Test (X) 
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Figure 6(c). Subpopulations with the sanve standard 
error of estimate and the same slope 
but different intercepts. 
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variable and a criterion variable for one subpopulatxon. The condi- 
tioaal distribution of Y (criterion) given X (predictor) is assumed 
to be normal. The shaded portion of the distribution represents the 
risk level for a particular value x on the test. In Figure 6(b), 
the regression lines for the two subpopulations coincide; hoyever, 
the standard error of estimate is smaller for subpopulation tt^ than 



for "n^. Provided, as in the figure, y < y# (the sample jmean) , then 

for any test score x, the level of risk is less for members of 

T 

subpopulation tt^ than for members of subpopulation tt^. Thus, if 

all applicants with predicted criterion scores greater than or equal 

to y* (X >^ X*) were selected, the test would discriminate against 

subpopulation tt^^ according to. the Equal Risk Modef^ In thj^ situation, 

to make the selection procedure fair (according to the E^u^l ^^^V 

Model), members of subpopulation tt with test scores greater than or 

1 »* 

equal to x* would be accepted and members of subpopulation tt with 
test scores greater than or equal to would be^aqcepted. However, 
if the standard error of estimate had been the same in each subpopu- 
latlon or if y* *= y., then the use of a single cut score would be 
considered fair to members of both subpopulations. In Figure 6(c), 
the two subpopulations have the same standard error of estimate and 
the same slope b^t different intercepts. For any test sQore x, the 
level of risk is less for a person from subpopulation tt^ than for a 
person from siibpopulatlon tt^. /If a single cut score is used, th€;n%he 
test discriminates against members of subpopulation tt^ according to 
the Equal Risk Model. The selection procedure would be considered fair 
(according to the Equal .Risk Model) ^ if ^nemberfi of subpopulation tt^ 
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(1^2) with test scores greater than or equal to x* (X2) are accepted. 

Note that If each stjbpopulatlon has the same standard error of estimate 

f 

and the same slope, then the selection strategies proposed by the 

Regression Model and the Equal Risk Model are the same. The two models 
are very closely related, differing only in that one is based oh 

the mean-sguared-error criterion an^ the other on a threshold-loss 

» ♦ 

criterion, and both are straightforward applicatilpns of statistical^ 
decision theory. j ^ 

\ . . 



The converse of the Equal Risk Model would require, given a 
minimum level of satisfactory criterion perforniance (y*) that the 
predictqr cut scores* x* (i « 1, g) he determined so that 

Z « Prob(Y < y*|X « x*, ir^) « ... ».Prob(Y < y*|x « x*, ir ) , 



V 

/ (9) 



where- .Z is a fixed constant degree of risk for all subpopuldtions ir^^. 
This relationahipT can be rewritten as 

Z « 1 - Prob(Y _> y*|X =' x*, ir^) 

« 1 - z , \ ^ 

where Z » *Prob(Y >. y |X » x. , it. ) is the value to be equated among 
subpopulations for selection faiirness as specified by the Equal Risk 
Model. Thus, the criterion of the Converse Equal Risk Model is a linear 
function of that of the Equal Risk Model. Hence, unlike the Constant 
Ratio Model, the Conditional Probability Model and the Equal 
Probability F^pdel, the Equal Risk Model and Its converse will always 
specify the same selection strategy. There is no internal contra- 
diction; the model is coherent. 
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A Critique of the Constant Ratlo> the Conditional Pfobablllty 
and the Egtial Probability Models 

One problem \^lx the Conditional Probability Model and the 
Converse Conditional Probability Model is that each model treats onlyi 
one aspect (selection-success) of the ctpj.ture~falr selection issue. Recall 
that K « Prob(X >, x^|Y >. V » it^') K = Prob(X < x^|Y < y , ir^) are the 
values to be equated among subpopul^tlons for selection taimess as 
specified by the Conditional Probability Model and Converse Conditional 
Probability Model, respectively. In practice, we mui&t consider 
equating both K and among subpopulations^ Since It can be sh^wn ^ 
that only under certain special conditions equating K amohg aubpopu- 
lations leads to eqipting ^ among suJ>j>opulations ,feand vice versa 
(refer to the section entitled The Cpnverse Conditional Probability 
Model>» it might be, suggested that in order to take both aspects of 
the culture-fait selection issue [the conditional probability of selection 
given succesa/^KT^d the conditional probability of rejection given failure 
(K)] into coAsideratlon, we should at least contemplate equating some 
combination of K and K instead oi trjring to /equate, independently, 
either K or K among subpopulations . Howevei, it will be difficult to 
decide what function of K and K should be equated among subpopulations 

far fair test use. / . , ■ ■ _ ■ ■! f 

Similar comments can be made regarding the Constant Ratio and 
Converse Constant Ratio models, and regarding ' the Equal Probability 
and Converse Equal Probability models. Each model deals with only one 
aspect of the culture-fair selection, issue.' In contrast, the'Hef inition of 

39 
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cuiture-'fair selection proposed by the Equal Risk Mc^l deals wit^ bbth 
sides of the issue ^ because if one equates Z [EquatJ^ ^8)] among subpopu- 
lations, then one also equates the converse Z- = 1 ~ Z [Equabion (9)1 
among subpopulations. * ' . » 

To see why one should consider both aspects of the issue of • 
falmesp, note that if one tries to Increase thei conditidnal proba- . 
^ bility of selei^tion given puccess (K) > then one will decreajsje the 
^conditional probm)ility of rejection given failure (K). Rewrite K 



and K as follows: 



and 



Prob(X X., Y 1 y [tt ) 

vK«— ^ 

^ Prob(Y £ y jir ) 



_ Prob(X < X*, Y < y*|7r ) 
K « - 



r 



Prob(Y < y Itt^X 



Now, for a specified minimum level of criterion performance (y . 
Prob(YV y (tt^) and Prob(Y < y |tt^) are fixed values. Thus, fp^ K to 
increase, Prob(X 1 ^ >y K^) ^^^^ increase implying that the 

ic * 

predictor cut score (x^) must decrease, (See Figure 3.) It is 

5fc ' ' ic ic t 

then clear that if x^ decreases, Prob(X < x^, .Y < y |7r^T must 

decrease implying that K must decrease. Hence, although both lar^ge 
K and large K seem desirable for a given subpopulation tt . , any • 

if 

predictor cut score x^ which leads to ^n increment in K will, result 
in t decrement of This is similar to the situation in hypothesis 
/testing where one tries to avoid two types of errors and, therefore, 
.to reach a compromise iff^electing/a critical region. ' Thus, if 
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. one Is inclined to build a model around Cole^s conception of selection 
fairness, then one must 'try to equate a function of K and K among 
subpopixlations rather than to equate K or K alone • To do this 
would require a valiie specification for the relative' size of K and K. 

One can also show that ^ the case of the Copstant Ratio Model 
and^the Converge Constant Ra;tio Model, R [Eq^uation (5)] will 
decrease as R [Equation (2)] increases. Thi^ indicates the same 
dilemma of trying compcomise between equating R or equating R 
among siibpopixlations ^ Thus, a definition of selection fairness can only 
be satisfactory if one considers both R and R. ^Thus, among the 
Cons^s^t Ratio, the Conditional Frobability, the Equal Probability 
and the Equal Risk models, only the Equal Risk Model is satisfactory 
in the sense that it takes both sides (selection-success and rejection- 
failure) of the culture-fair selection issue into account. 



The Culture-Modified Criterion Model 
In addition to the criterion variable Y and the predictor 
variable X, Darlington (1971)' defti^^s a third variable C, which 
denotes an applicant's group membership. The variable C may be , 
either dichotomous or continuous (|s^g., sex; race; socio-economic 
•status). Darlinfeton then givers (and discards) four definitions of 



41 



cultural fairness in terms of the correlations among the 
three variables X» Y» and C. 

In order to istate the four definitions in common correlational 
/tenttinology, simplifying assumptions are introduced: the valuables 
X and Y have a blvariate normal distribution in each subpoptHation; 
the correlation between X and Y (r^) is positive; and the standard 

devlatlon on the test (s ), the standard deviation on the criterion 

h 

-(s and are constant for each subpopulation. Darlington's 



four def^iltions of cultural fairness are: 

(3) .r^- r^yr^, ana'' • 

(4) r^,-0, ' . , . ^ 

where ttie^r's represent the correlations between the subscripted , 
variables. "In e$ch case» a t^st Is considered culturally fair if 
it satisfies the appropriate equation. (Darlington^ 1971» p. 

De£lnlt:ion (1) Is equivalent to the Regression Hodel v^ich 
^requires a common ^regression line. Definition (2) Is the same as 
Thomdike*s Constant Ratio Model. ^ Definition (3) is a special case~ 
of Cole's Conditional Probability. Model. Definition (4) is the same 
as the requirement that subpopulations ^have equal means oiy the test. 
(Darlington, 1971, pp. >3-75; Linn, 1973, pp. 156-157.)^ 
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The fotir definitions yield contradictory results except in the 

case of perfect validity (r = 1) o.r in the case of equal subpopu- 

lation means on the criterion (r ° 0). Darlington also clAims that 

cy 1* 

the four definitions are 

all based on the false view that optimui^ treatment of 
cultural factors in test construction or test selection can 
be reduci^d to completely mechanical procedures. If a 
conflict arises hetween the two goals of maximizing a testes 
validity and minimising the testes discrimination agai^t 
certain cultural groups, then a subjective, policy- level 
decision must be made concerning the relative importance of 
the two goals* [p, 71/] 

Darlington then suggests that instead of predicting the criterion 

variable Y that a variable (Y - kC) be defined where k is determined 

by a stibj active value judgment on the part af the decision maker 

(test user). Darlington urges that 

the term "cultural fairness" be replaced in public discussions 
by the concept of "cultural optlmality/' The question of 
whether a test is culturally optimum can be divided in^two: ' 
a Slab j active, policy- level question concerning the optimum 
balance between criterion perfoirmance and cultural factors 
(operationalized as the optimum value of k), and a purely 
empirical question concerning the testes correlation with 
the culture-modified variable (Y - kC) and whether that 
correlation can be raised. , [pp. 79-80.] 

According to this formulation, each institution must first choose a 

value of k, indicating whether there is special value in the selection 

of members from some subpopulation* That is, the decision maker must 

answer the question, "How many units on Y are considered equivalent 

in value to one unit on C?", Then, the psychometrician's job is to 

contruct a test to predict the variable (Y - kC). Note that when k 

is set equal to zero, that is, when there is uo reason to favor one cultural 

group* this procedure reduces to that of the Regression Model. Also 
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note that where the other models for culture-fair selection would set different 
predictor cut scores for each subpopulation, Darlington would add a 
specified number of points to the scores of pne subpopulatioo and 
then use the sam^ predictor cut score. 

Darlington^s formulation of selection fairness recognizes, explicitly, 
that the variable which is traditionally considered to be the criterion 
(e.g., college grade point average)* is not the only criterion. Group 
membership or culture is also part of the criterion. Darlington, 
then, argues that the traditionally - accepted criterion must be 
, modified for culture; hence the reference to Darlington* s formulation 
of selection fairness as the Culture-Modified Criterion Model. 

An Appraisal of the Test Bias Models 

The Regression, the Constant Ratio, the Conditional Probability, 
the Equal Probability, the Equal Risk and the JJulture-Modif fed Cri- 
terion models are each explications of general concepts of what 
constitutes the fair use of tests in a selection situation. There 
3eems to be nothing in the literature that clearlyL^indicates, when, 
if ever, one of the models is preferable to the other five models. 
Thus, the practitioner has ho clear guidance in the choice of a 
culture-fair selection model. Further, we have suggested that the . , , 

^onstant Ratio, the Conditional Probability, the Equal Probability 
models and their converses are internally contradictory . 

There has been considerable interest in the Constant Ritio 
Model and the Conditional Probability Model based on the fact that 
these models yield a^ popular result, in that they apparently give 
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lower cut scores for disadvantaged minority populations* The appea-I 
of'^lthese models, then, is that they produce^^ desirable result. 

However, we might well contend that it is generally not appro- 
priate to evaluate the correctness of a model. solely dn the basis 
of the pleasantness or unpleasantness of its implications but, rather, 
that one must look carefully at the logical structure of the model. 
One must be sure that the'model is getting the right restilts for the 
right reasons. If 'the models are giving' the right results for the 
-^ong reasons, it may well be possible that, in some other circum- 
stances, wrong answers will be forthcoming. We shall see that^iftis 
is indeed the case. We shall show that these models sometimes can 
produce most undesirable results and could, iti fact, be used to 
justify discrimination against some niinority groups. 

To see that this may happen, consider a situation in which the 
regression lines in the minority and the majority populations are 
identical, but in which the mean* values of X and Y are higher in ^ 
the minority (disadvantaged) population and lower in the majority 
(advantaged) population. (Refer to Figute 4.). This situation is not 
typical but, in fact, can be found if one compares,^ for inslfance, a 
Japanese-American minority population with the Anglo majority popu- 
lation. In this situation, Voth the Constant Ratio and the Condi- 
tional Probability models will give lower predictor cut scores and 
hence easier entry to the majority population. The Regression Model 
and the Equal Risk Model will give identical predictpr cut, scores. 
If, as well may be the case, the Japanese-American subpopulation 
has been discriminated -against in some situations and is thus a dis- 
advantagai group, then our desire might be to^provide easier access 
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for that subpopulation, but, in fact, the two madels being considered make 
access more difficult.^ 

Protn this example, it can he seen that the ,tWQ models being discussed 
make a correction that is^^sually in the desirable direfction, but that they 
make that correctipn for the wrong .reason. They make the correction simply 
because of differences in the mean values of X and Y in the two populations 
and they only coincidentally take into account the public desire or social 
necessity to rectify unfair treatment to a minority population. The degree 
of advantage to the minority population is largely a function of the differ- 
ence in the mean test scores and in no way directly reflects the degree of 
prior discrimination and disadvantage that group has suffered. On the other 
hand, if, following the general ideas to be laid down here, on^ allows that 
differential treatment should be given, to a degree agreed upon by the poli- 
tical process to some heretofore disadvai^^aged group, then a lower predictor 
cut score will be obtained for that group. In this case, the lower score is 
obtained for the right reason, because of their disadvantaged status (different 
utility structure), and not simply because of a difference in mean values. We 
judge that for these reasons the use of the Constant Ratio, Equal Probability 
and Conditional Probability models and their converses is contrai'ndicated. In 
stating this, it is not suggested that any ill effects will necessarily result 
from their use* Only by more detailejd study would, it be possible to document 
more completely all of the situations in which these models break down. However 
one could easily name other minority groups, discrimination against which would 
be. sanctioned by any official, or other, recommendation, endorsement or adoption 
of any of these models. Finally,' we would remark that there is no reason to 
bqlieve th^t either the Conditional Probability or' Constant Ratio models will 
provide, under any circumstances, the degree of compensation that I, think 
^ certain disadvantaged^>er^ns should have.. 

• * 



Maximizing Expected Utility . 

There ©ci^ts a Body of quantitative reasoning, whose origins 

jaxe ancient anff^remote, that has received codification in this 

^ ^ , - * . 

c^tury in the ^ work of Von Neumann and Morgenstem (1947), Wald 
(1950), and others. In the theory of the tational-economic man, 
developed in these writings, when all probabiljjties of outcomes 
are assumed known (an assumptlon"15aa^ explicitly here and impll- 
city in previous statements of the models under consideration) 
there is a simple paradigm required for rational decision. In 
that paradigm the desirability or utility of each possible outcome 
is stated quantitatively* Then, given ill available Information 
concerning the person in question, the probability of each possible 
outcome is stated for each decision u^der consideration. Next,, for 
each possible decision, the jitility of each outcome 10 multiplied^ 
by the probability of each outcome and the products are summed to 
provide an expected utility . Finally, that decision is then made 
for which the expected utility is highest. Most: statisticians 
Interested in decision problems accept the correctness of the ^ 
,Von Neumann and Morgenstem-Wald model and the incorrectness of 
any statistical decision procedure that does not conform to that 
model* It seems clear that the Constant Ratio Model, the Conditional 
Probability Mod^l and the Equal Probability Model do not conform to 
that model) thqugh the ideas that are at their bases may well be ^ 
reformulated Ixt a coherent mianner. 

If the utility of an outcome dependably on the individual 
cmtcome (success-failure) and the subpopulation involved, then the 
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axloma of rational decision-iaaking require the maximization of 
expected utility (but see Suppe8,^197ii,yfor a slight weakening of 
this statement that is not relevant here). That maximization process 
involves only the conditional probability of success given test sco^e 
and the utility of that success. , It doifef-.not involve any o^f the' 
marginal or conditional probabilities used by Thorndike, Cole or 
Linn. From this ppint of view, the fundamental fallacy in eacH of 
these mo<fels is that they are based on the wrong conditional prpba- 



Specifically, the conditioning process must be on the specific 
value X observed on the person ahd not on the marginal distribution 
of X (Thorndike), t;he conditional Jiistribution of X given y (Cole) 
or the eveiit X^x (Linn). The thr^^^^m^jcfels mentioned above do not 
use the correct probability. The Regression Modt-^l an(| the Equal . 
Risk Model do, and it is for this reason that no logical contra- 
dictions have arisen with these models. This is not to say that 
these latter models are entirely satisfactory; indeed. You could 
judge them to be generally unsatisfactory but only because You judge 
the utilities they adopt to be inappropriate. While these models 
are both special cases in the general decision-theoretic formula- 
tion they are, it would seem, much too special. The utilities they 
adopt, implicitly, are not particularly compelling and Unfortunately 
.they do not make these assumption^ very clear. More general models 
are required. - . . 

Some individuals (e.g., Humphreys, 1973) have indicated a^ 
basic dislike of differential treatment of groups while possibly 
accepting its short-term desirability. That position has merit, 

'48. 
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though in the current climate of opinion, It may represent a minority 
view. One coherent expected' utility .piodel (the Equal Risk .Model) 
provide^ for equal treatment of all persons. However, we suggest that 
if this criterion is appropriate it would be better to arrive at it 
. on the basis of careful analysis ahd debate in the area of public 
policy rather than because of some notion regarding the universal 
applicability Of' any one model. . , • ' 

Thorndike has argued forcefully that, the marginal distributions 
of both X and Y are Important in cult^ure-fair' testing whereas the 
decision-theoretic formulation ye discuss in the next section con- 
centrates pnly oii the conditional distribution of Y given x* 
(thorndike* s view, which is that some consideration must be given . 

.ln*the setting of cutting scores to their effect on tlie percentage 

> / ' . ^ 

of successful pexsoiis In^eaAh subpopulation, dan possibly be accommo- 

dated within a decision-theoretic framework. ) It would Qeem to us ' • 

that it might be appropriate' in assessing* utilities in the , two .sub- ' 

populations to take into cofaslderatlAn the"" ImpiicatjLons with respect 

to these maEginal^distribution^. We would expect, however, this - 

consideration' of marginal distributions to also take into account 

the effect on the* percentage of failures in the two sqbpopulations. 

Such investigation could result in our utilities being related t<> 

the location parametets ,of the marginal distribiiitlons,. but this woul4' 

not; affect the probability aspect of th^ decisipn-theoxetic .formula- 

tiott which would still depend pnly on the UisCributipn of Y given x. 

A. similar reiiiark;mlght be made with .pBspett td Cple's Conception of^ 

culture-fair selection which might bi^-l^ef pTtQulated ip terms of utili- 

ties rather than prabablll ties. However,' in our Judgment ,'^un til 
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these Ideas are given a strict decision-theoretic formulation In 
terms of utilities that You and I can understand, we must caution 
against any use, adoption or recommendation of these ideas. 

Darlington's Culture-Modified Criterion Model 1^ the only model 
surveyed that addresses itself to the utility question. It also has 
the desirable feature of focjusing on the correct coindltlonal proba- 
bility. Unfortunately, this formulation is still not entirely con- 
sistent with the decision-theoretic approach (i.e., it does not 
incorporate a formal utility function), and hence may not be accept- 
.able. We would' urge Darlington to restate his model so that it can 
be evaluated more easily. 

The Threshold Utility Model 

In ^ recent paper^ Gross and Su" (1975) investigated one ddcisipn 
theoretic approach to the culture-fair selection problem. This fprmu 
lation maximizes expected i^lllty using a threshold utility model. 
The discussion in this section is taken from a somewhat more complete 
analyses due to Petersen (1974). 

Assume that the applicants to an institution can be separated 
into two subpopalatlons referred to as the minority population (tf^^) 
and the majority population (1^2) • Further, assume that a predictor 
variable X (a test score) and a criterion variable Y (a measure of 
performance) have possibly different (unspecified) positively 
correlated blvarlate distributions in each subpopulatlon. 

Each applicant to the institution will, if accepted, be either 
a successful performer or an unsuccessful performer. The state of 

'the applicant being a successful performer is represented by 

/ * 

those values of Y greater than or equal to the point y , and the 
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state of the applicant being an unsuccessful perforaer Is 

(' 

represented by those values of Y less than the point y* where ^* Is 

'* *^ i 

the minimum le'^l of satisfactory performance on the criterion' 

varl.iabie "k. , Two possible actions are open to the Institution.' The 

J'* ' , 

Institution can either accept the applicant or reject the applicant.' 
The accept decision Is represented by those values of X which 
are greater than or equal to the point x*, and the reject decision 
by those values of X which are less than the point., where Is • 
the cut score on the predictor variable X^ and may differ for each 
subpopulatlon tt^, 1 = 1, 2. 

For each subpopulatlon tt^, the action decided upon by the 
institution can have one of four outcomes: , 

Oj^: X >^ X* Y >^ y* ^ applicant is accepted and is 
successi^ul. 

X < x^^ Y >. y. Aiv applicant is rejected but would 

have heen successful. 

• * * * ' 

0^: X < X. Y < y An applicant is rejectefd and would 

have been uxpuccessful. 

^ *" * ' 

0^: X >^ x^ Y < y An applicant is accepted but is 

laisuccessful. 

Outcomes 0^^ and 0^ are desirable since they represent correct 

decisions y whereas outcomes O2 ^"^^ 0^ ^re undesirable sjLnce they 

represent incorrect decisions. Up to this point the formulation^ 
is the same as for the constant ratio^ conditional probability, 

equal probablll|:yi and equal risk models. 

With these outcomes in mind, the institution designated ^ 

threshold utility function u(0j|7r^), j « 1, 2, 3, 4, ^or each 

subpopulatlon ir^, 1 • 1, 2, as defined in Figure?. For the 
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Figure 7 
Threshold Utility Fimction 



u(0^|Tr^) 



Y > y (successful) 



Y y (imsuccessf uiy 



'4 • 



ic ^ 

. X < (reject) X >^ x^^ (accept) 
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subpopulation tt^, the utilities u(0^|-7r^)^nd « tiYOjjir^) 

are associated with cortect decisions and should be larger than 

the utilities « u(02|7r^) and d^ J^i^i^\'^^ • The zero ^oint^is 

.arbitrary, though it may be convenient to take b^, d^ <^,0 and 

refer to these as disutilities. The degree of preference givert ai>y 
group will depend upon the utiliti^for th^t group. 

Regardless of the otciering of the utilities,^ the expected 



r 



utility* of selection for an applicant from subpopulation tt is 

• < . , i 

given by * / , '^^ ^ . 

^[u(0|7r )]= I uCoJtt )Prob(0j7r ) " ^ 



«,a^[Prob(0^|7r^)] b^[Pi;ob(02|7r^)l 

^ - + c^EProbCO^lTT^)] + dJProb(0^|7r^)] . (D 

Now, the process of selection is vlewed^as a series of separate 
decisions, each of which involves one applicant* Thus, the expected 
utility of the selection process is found by summing the expected 
utility of selection for an applicant [E^quation (1) ] over all 
app^licants, that is-, 

2 

Z 
1=1 



(g [u(0)] « Z p^{<^ [u(0|7r^)]}' 



2 4*' ^ 

Z P. Z.u(0J7r.)Pro{>(0 Itt.) , 
/i«l ^ i=l ^ ^ ^ ^ 



where p^^^ is th^ proportion of the combined applicant populatiqn (tt^^ 
and lr^) who are members of subpopulation tt. • The problem is to find 
predictor cut scores x^^ 6nd vi^ such that the expected utility of the 

r- 




selection process [Equation (10)] is a maximum * The .method for 
doing this is given 'b^ Gross land Su iand by Petersen. 

In; effect what this mgcfel does is to select those applicants 
"^^^th tlj^j^jghest expected, utility^selection. The utilities wtH , 

typically differ for different subpopulations. (If they do not,^ 
. we have the Equal Risk Models) The prediction equations may 'also 
differ in the subpopul'ations. 

The advantage of this model is that it requires ^ explicit 
public stajtement of utilities for each subpopulation. If 1 do not 
like the utilities You prpvide, \an irixon^ive public debate will 
no doubt ensue; with neither of tis plalming any axiomatic justifi- . 
cation for our utilities. The Important thing is ^hat the discussioh 
Ke public with all interested parties participatjing in the debate. 

In our judgment, the .concepts of culture>-fairne68 and group . 
parity are neither useful, nor tenable, and the models spawned from 
them should xlot enjoy itfetitutional endorsenf^nt. The problem, we' 
think,, should be reconceptualized as a 'problem in^ maximizing expected 
utility. The Threshold Utility mbflel is one possibly^ useful model. 

It has its limitations. The concerns that motivated the Thorndike, 

A * ^ ' 

^ Cole and Linn models are ^Important ones; however they can be explicated 

in conceptually slmpl'fe extensions of the Threshold Utility model. 

This might involve dropping the additivity assumption of this model, * 

. Our main purpose in this paper has been' to show that the ideas of. 

, cuTfure-fairness and group parity have spawned incoherent models that 

can sanction the v^ry discrimination they seek to rectify. Any new 

explications of these ideas will^need to be scrutinized carefully from 

statistical, .psychological, social, ethical and legal points .of view. 
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